Trace Solution Paths for SVMs via Parametric Quadratic Programming

The performance of SVM models often depends on the proper choice of their regularized parameter(s). Some recent researches have been focusing on efficiently building all SVM models against the regularized parameters, thus defining a task of tracing the regularized piecewise linear solution path for SVMs. It has been widely known from an optimization view many SVM types can be formulated into quadratic programming. But it has seldom been pointed out the pathtracing for SVMs can be attacked by parametric quadratic programming (PQP). We from a PQP view consider the relation between path-tracing for SVMs and the generalized mean-variance portfolio optimization. The interdisciplinary relation allows us to handle the path-tracing task by tailoring the critical line algorithm (CLA) originally for meanvariance portfolio optimization. Although the CLA shares much in common with existing path-tracing approaches, it utilizes more systematically the equality and bounding constraints in the PQP formulation, and hints us to devise a robust one-per-iteration approach based on Karush-KuhnTucker conditions. Moreover, under the unified treatment powered by PQP and CLA, we can further derive ways to overcome pathological cases caused by empty or singular linear systems. In experiments we show the power of efficiently generating all models for one-class support vector classification and typical support vector machine.